DOCUMENT RESUME 



ED 409 427 



CE 074 342 



AUTHOR 

TITLE 

INSTITUTION 
REPORT NO 
PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 
JOURNAL CIT 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Walker, Elizabeth 

Appraisal in FE- -Where Are We Now? Developing FE. 

Further Education Development Agency, London (England) . 

ISSN-1361-9969 

97 

95p. 

Further Education Development Agency, Publications Dept., 
Mendip Centre, Blagdon, Bristol BS18 6RG, England, United 
Kingdom (10 British pounds) . 

Reports - Research (143) 

FEDA Report; vl n9 1997 
MF01/PC04 Plus Postage. 

Evaluation Criteria; Evaluation Methods; Evaluation 
Problems; *Evaluation Utilization; Foreign Countries; 
National Surveys ; *Personnel Evaluation; Postsecondary 
Education; Questionnaires; *Systems Approach; Tables (Data) ; 
*Teacher Evaluation; Technical Institutes; Vocational 
Education 
*Great Britain 



ABSTRACT 



The extent to which staff appraisal has been implemented at 
Britain's public sector further education (FE) institutions and the 
characteristics, benefits, and disadvantages of existing appraisal systems 
were examined in a survey of 233 FE institutions (36% of all institutions in 
the public FE sector) . Of the FE institutions surveyed, 76% had one appraisal 
scheme, and 18% had two or more. Of those systems, 57% were introduced in 
1992-1994 and 37% covered all full-time staff. FE institutions were using 
appraisal systems to identify and implement appropriate staff development and 
identify targets related to organizational objectives for staff development. 
Appraisal was almost completely unrelated to performance- related pay. Most 
appraisal systems reflected principles of good practice in terms of 
development, design, and implementation, and most were separate from 
disciplinary, grievance, capability, promotion, and reference procedures. 

Most monitoring and evaluation systems focused on individuals rather than the 
organization. Appraisal systems were praised for contributing to staff and 
organizational development and providing opportunities for dialogue between 
appraiser and appraisee. The most commonly cited disadvantage of appraisal 
systems was the heavy time commitment involved (especially for appraisers) . 
(Eleven tables are included. The report contains 26 references, a 48 -item 
bibliography, the survey questionnaire, and 31 figures.) (MN) 
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Executive summary 



More than one-third of all public sector further education (FE) 
institutions have been surveyed to identify the extent to which staff 
appraisal has been implemented. The findings have revealed the main 
characteristics of appraisal schemes and their perceived benefits and 
disadvantages. This report presents the findings, and considers their 
implications for the FE sector, and in the context of relevant literature. 

For the purposes of this report, appraisal is defined as: 

An opportunity to review and discuss with each individual his/her past 
performance and, based on the conclusions reached, agree a plan of action 
and/or priorities for the forthcoming period. (Edwards, 1984) 



Research questions 

Current literature on appraisal was consulted to identify the key issues, 
which would then become the focus of this research. Chapter 1 gives the 
details. The research questions which emerged from this review fall into 
three categories: 
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Extent of implementation in FE 

Given the varied pattern and history of appraisal in the FE 
sector, and the introduction of the National Joint Council 
(NJC) framework for academic staff in 1991, to what extent 
has appraisal been implemented? 

Characteristics of appraisal schemes 

- Strategic context 

What do FE colleges see as the most important purposes 
of an appraisal scheme? 

What is the relationship between the purposes of specific 
schemes and organisational objectives, in particular the 
overall aim identified by the NJC: 

to enhance the quality of the service provided by the institution? 

- Design, implementation and evaluation 

In the context of good practice, to what extent are the 
design of the scheme and the processes of implementation 
and evaluation likely to contribute to or detract from the 
specific purposes and NJC overall aim identified above? 

Effectiveness of appraisal 

What are the perceived benefits and disadvantages of 
appraisal schemes and to what extent do they fit with 
identified purposes and the overall aim? 

Is there evidence of ongoing development of schemes to 
continue to improve their effectiveness? 
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Key findings 



Extent of implementation 

• A total of 272 schemes in 233 further education institutions 
were surveyed. These institutions constitute 36 per cent of all 
in the public FE sector. The results reveal widespread 
implementation of appraisal: 76 per cent of institutions have 
one scheme, and 18 per cent have two or more schemes. 

• The majority (57%) of current schemes were introduced in the 
period 1992-1994, which is the period agreed by the NJC. 

• Out of all of the schemes, 37 per cent cover all full-time staff; 
in all other cases only particular categories of staff (such as 
academic or business support) are included. 

Characteristics 



Purposes 

• FE institutions use appraisal schemes to identify and imple- 
ment appropriate staff development, and to identify targets 
for this development, linked to organisational objectives. 

• There is almost no evidence of appraisal being linked to 
performance-related pay. 



Links with organisational objectives 

• Findings suggest that appraisal schemes can make a strategic 
contribution to the quality of service provided by the 
institution, as envisaged in the agreed national FE framework. 

• A high percentage of FE institutions (65%) are committed to 
the Investors in People Standard and a lower proportion 
(30%) have been awarded the standard. 
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Design 

• Most of the schemes reflect principles of good practice in 
terms of their development, design and implementation. 

• Appraisal schemes are separate from disciplinary, grievance, 
capability, promotion and references procedures. 

• A high proportion of schemes (83%) use a line management 
model. There is evidence of peer appraisal in eight per cent of 
schemes, but very little evidence of 360 or upward appraisal, 
which each occur in only two per cent of schemes. 

• The main additional input is from other managers (39% of 
schemes). There is some input from subordinates (18%), 
students (15%), other college clients (7%) and peers (27%). 

• Observation of teaching contributes to appraisal in 64 per cent 
of schemes, and evaluation of tasks contributes to appraisal in 
54 per cent of schemes (8% gave no response). 

• Other evidence which informs appraisal includes the 
outcomes of previous appraisal interviews (70% of schemes), 
inspection feedback (29% of schemes), section reviews (30% of 
schemes), and course reviews (9% of schemes). 

• The majority (76%) of schemes have appeals procedures; these 
are only known to have been used in 10 per cent of schemes. 



Implementation and evaluation 

• The focus of monitoring and evaluating appraisal schemes is 
at the individual rather than the organisational level. 

• For most schemes (83%), participation is compulsory. 

• Appraisal is annual in almost half (48%) of schemes and takes 
place every two years in most other schemes. 

• Just over half of the schemes (57%) include an interim review. 
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• There is little evidence of time specifically allocated for 
appraisal, for appraisers or appraisees. 

• There is some choice of appraiser in 56 per cent of schemes, 
although in most cases this is constrained. Just more than a 
quarter (27%) of schemes do not allow any choice of appraiser. 

• Appraisal training is provided in the majority of schemes, 
most commonly one day for appraisers and half a day for 
appraisees. Training is compulsory in the majority of schemes; 
appraisers and appraisees are trained together in 29 per cent 
of schemes and separately in 43 per cent of schemes. 

• In the majority of schemes, preparation for appraisal 
frequently includes pre-meetings and the exchange of 
documents between appraiser and appraisee. 

• Common appraisal locations include the appraiser's office, 
specific rooms, and, less commonly, the appraisee's office. 

• The majority of schemes use a range of standard documen- 
tation, the use of which is generally compulsory. 

• Confidentiality of appraisal documentation is regarded as 
highly important. In the majority of cases only the appraiser 
and appraisee have access to the full appraisal record. 

• The identification, recording and monitoring of specific 
outcomes are common features of appraisal schemes; 
although the level of detail recorded is variable. 

• Nearly 90 per cent of schemes are monitored to check that 
appraisal is taking place. Monitoring to check that outcomes 
have been identified, implemented and reviewed is less 
common, although it is reported to occur in 50-67% of schemes. 

• Out of all of the schemes, 40 per cent have been evaluated to 
identify benefits to individual appraisees. There has been a 
substantially lower level of evaluation of benefits for the team 
(7%) and for the organisation (14%). 
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Effectiveness 



Benefits and disadvantages 

• Cost-benefit analysis of appraisal has only taken place for 
seven per cent of schemes. 

• Schemes are perceived as having more benefits for individual 
appraisees and appraisers than for teams and whole organ- 
isations, although benefits are identified at all of these levels. 

• The benefits most frequently identified reflect the explicit 
purposes for which schemes have been designed, that is to 
contribute to staff development and to identify targets linked 
to organisational objectives. The opportunity for dialogue 
between appraiser and appraisee (usually line manager and 
staff member) is frequently identified as a benefit of appraisal. 

• The most commonly identified disadvantage of appraisal 
schemes is the heavy time commitment involved, particularly 
for appraisers. 

Ongoing development 

• The findings reveal that many of the institutions (51%) are 
planning changes to schemes. 

• The most commonly planned changes include making schemes 
simpler and linking appraisal to organisational objectives. This 
suggests an awareness of the need for developments in this 
area, particularly in the context of the commitment of a high 
proportion of colleges to the Investors in People Standard 
which takes a strategic approach to staff appraisal. 
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Chapter i 

Development of appraisal schemes: 
key issues 



Developments to date 

The FE sector workforce has, for historical reasons, comprised man- 
agement staff, staff described as 'academic' (those in lecturing and in 
some research posts), and staff described as 'support' (those in adminis- 
trative, technical and clerical posts), referred to as 'business support' for 
the purposes of this research. Different terms and conditions have applied 
to these groups, including different arrangements for appraisal. At the 
time that most appraisal schemes were developed, management staff in 
general fell within the scope of the appropriate appraisal arrangements, 
depending upon their particular role and the individual college 
approach. For business support staff, the appraisal schemes developed 
with trade unions in local education authorities (LEAs) have been 
applied to staff in some, but not all, FE colleges; some are compulsory. 

In 1991 a national framework for appraisal of academic staff was agreed 
by the National Joint Council (NJC), which included employer (LEA) 
and academic staff union representatives. This came about following the 
1986 Education Act which required: 
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local education authorities and others to secure the regular appraisal of teachers 
in schools and further education establishments. (Section 49, Scribbins and 
Walton, 1987) 

Colleges were to develop local schemes by August 1992 and to complete 
the first year of the appraisal cycle for academic staff by 31 August 1994. 
Some colleges opted for a single scheme for all staff while others 
concentrated on meeting the NJC agreement requirements of 
introducing a scheme solely for academic staff, to operate either on its 
own or alongside an existing scheme for business support staff. A single 
scheme would help to promote a shared commitment to organisational 
objectives. If introduced sensitively, it could have a positive impact on 
organisational culture, which would be critical to its effectiveness, 
suggest Long (1986) and Anderson (1993). One of the benefits of this 
survey would be to reveal the extent to which colleges have 
implemented single appraisal schemes to cover all staff. 

The range of approaches to appraisal in FE colleges is diverse. It is likely 
that these practices have diversified even further following 
incorporation in April 1993, when LEAs ceased to have jurisdiction over 
colleges. This survey will show different approaches taken in FE today. 

Identifying the key issues 

A review of current literature on appraisal has revealed the main issues 
surrounding staff appraisal in FE. These fall into three categories: 

• strategic context of appraisal: the relationship between the 
purposes of specific schemes and organisational objectives, 
and between appraisal and other organisational procedures 

• design, implementation and evaluation: including the appraisal 
model chosen, and the training, preparation, documentation 
and resourcing, monitoring and evaluation processes used 

• effectiveness of appraisal: perceived benefits and disadvantages, 
outcomes of a cost-benefit analysis and planned development. 
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strategic context of appraisal 

Taking a strategic human resource management approach to appraisal 
would suggest that the organisation intends the scheme to increase its 
ability to meet strategic objectives. Anderson (1993) refers to perform- 
ance appraisal as a 'key element' in enabling organisations to: 

evaluate, utilize and develop the skills and abilities of their employees to ensure 

that organisational goals are achieved. 

Opinions vary about how appraisal schemes might best be used to 
accomplish these aims. Some schemes have focused on evaluative 
approaches, while others have emphasised developmental approaches. 
These are sometimes referred to as 'hard' and 'soft' schemes (Scribbins 
and Walton, 1987). Table 1 (overleaf) shows the main purposes of 
appraisal identified in the literature, ranged along this dimension. 

The evaluative approach measures performance in some way and 
punishes or rewards people (for example, through pay) according to 
how they perform. It is based on the belief that extrinsic rewards are 
motivating and will make a positive impact on performance. The 
developmental approach is based on the assumption that feedback on 
performance and the ability to do the job to a higher standard will itself 
be intrinsically motivating. Herzberg (1968) suggests that intrinsic 
motivators, such as job enrichment, will have a more sustained effect on 
performance, while extrinsic factors such as pay are only likely to 
motivate in the short term. These two approaches have different 
implications for the design of appraisal schemes and for the focus of 
organisational resources. As a result, they are sometimes seen as 
mutually exclusive. The evaluative approach allocates more resources to 
those who perform well while the developmental approach allocates 
them to those who need to improve their performance. 

There are many appraisal schemes which aim to achieve both objectives: 
to set and review organisational objectives and to identify staff 
development needs. This philosophy underpins the Investors in People 
Standard (Employment Department, 1991), which sees identification 
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and communication of organisational objectives as a prerequisite to 
establishing an appropriate system for staff development. Other support 
for this approach comes from, for example. Beer and Walton (1990) and 
Hartle (1995). 

Focusing more specifically on the FE sector, the NJC framework states that: 

the overall objective of schemes of staff development, training and appraisal 
should be to enhance the quality of the service provided by the institution... 



More developmental (softer) approaches 



i 



t 



To provide opportunities for employees to discuss career ambitions 
with their appraisers 

To identify staff training and development needs 

To facilitate manager/worker dialogue 

To identify employee potential 

To contribute to quality initiatives 

To improve future performance 

To promote cultural change 

To provide feedback to appraisees from managers 

To monitor past performance 

To set performance objectives 

To help the company make decisions about pay 

To facilitate disciplinary action 



More evaluative (harder) approaches 



Table 1: Main purposes of staff appraisal 

[Sources: Anderson (1993), Gill (1977), Long (1986), Yeates (1990)] 
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Schemes of development, training and appraisal should ... allow for the proper 
determination of individual lecturers’ professional development needs and for 
the joint clarification of individual priorities and objectives. (Lee, 1991) 

Lee reports that the appraisal schemes of the seven FE colleges which 
piloted the NJC framework only fulfilled these intentions to a limited 
extent. His findings, published in an National Foundation for 
Educational Research (NFER) report, include the recommendation that; 

In order to benefit individuals and the institution, and thereby enhance the 
quality of the service provided, the links between individual approaches and 
corporate review need to be firmly established. (Lee, 1991) 

This approach recognises the developmental nature of appraisal, and 
emphasises the motivational importance of feedback and development 
rather than extrinsic reward or punishment (such as discipline, 
promotion or performance-related pay). By stressing the need to identify 
targets which are linked to organisational objectives, it also has a 
strategic focus. Such an approach is recommended good practice in 
much of the appraisal literature (for example Anderson, 1993, Hartle, 
1995) and is a key point of the Investors in People initiative 
(Employment Department, 1991). Speaking specifically of FE, Liz 
McGrath (1996) suggests that: 

A' blame-producing, problem-identifying appraisal system is not the only 
alternative to an over-cosy praise or 'celebration' chat. It is possible to be 
perfectly clear about organisational imperatives and constraints, about 
objectives and responsibilities, within a context which incorporates recognition 
of work, appreciation of effort and success, need for support and areas of 
potential development. 

Although widely recommended, such an approach does require a 
thorough and skilled attitude to the development, design and imple- 
mentation of the scheme in order to be motivational, credible and effective. 
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Design and implementation 



Models of appraisal 

Key issues when designing an appraisal scheme are the identity of the 
appraiser, the range of information used as a basis for appraisal, and the 
means by which it is collected. The main models are: 

• line manager appraisal 

• peer appraisal 

• upward appraisal 

• self-appraisal 

• 360" appraisal. 

The most common model is the line manager one (Anderson, 1993). This 
is relatively simple to understand and implement, and allows appraisal 
to deal with performance management within the overall line 
management relationship. Such schemes should provide line managers 
with valuable information about staff roles and perspectives, and 
increase the probability that appraisal will have a strategic dimension, 
provided that line managers are clear about strategic priorities. 

Fletcher (1993) describes peer appraisal as having: 
an appeal in academic and teaching institutions. 

Evidence suggests that levels of use are relatively low, partly because of 
the reluctance of peers to move into this role, and partly because of the 
perceived need for management input into the process and its outcomes. 

Some element of self-appraisal is essential if appraisees are to be 
committed to the process. However, it is difficult to link such a scheme 
with other organisational processes if it is the only aspect of appraisal. 
Self-appraisal and upward appraisal (where those lower down in the 
management hierarchy contribute to the appraisal) are more likely to 
form part of the process than the whole of it. 
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Information from other colleagues, customers, clients and subordinates 
now has a higher profile in the appraisal process. This is due to the 
diverse management structures that exist within further education and 
to the increased emphasis on the link between appraisal and 
organisational objectives. 

Recognition of the importance of information from all angles has led to 
the development of the 360** model. Milliman et al. (1995) claim that this 
approach will increase the probability that employees meet their 
performance objectives. 

The extent to which information from students and other college cUents, 
and from course reviews and college inspection, informs the appraisal 
process is a significant issue to be addressed by this research. 

Degree of choice 

If appraisees are to take ownership of the scheme they need to believe 
they have some choice over appraisal decisions. If participation was 
optional then evaluation and development of employees would be 
piecemeal and the scheme's effectiveness as part of a strategic approach 
would be limited. However, a rigid scheme imposed on staff with no 
consultation would be demotivating and so also limited in effectiveness. 

Good practice suggests the need to balance flexibility with structure, 
taking into account the critical issue of organisational culture (Anderson, 
1993). Allowing some choice of appraiser, and including a supportive 
appeals procedure, will help to increase employee commitment. This 
will also help to address the issue of indirect discrimination - given that 
most appraisal schemes are line management models, appraisers are 
more likely to be white, male and able-bodied. 

Employers must be aware of this issue early on: when designing and 
implementing their scheme they should remember their legal 
responsibility under the Sex Discrimination Act 1975 and the Race 
Relations Act 1976 not to discriminate (My land, 1992). 
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The NJC has recognised that choice of appraiser is a sensitive issue. It 
has identified criteria which the appraiser must satisfy, which include 
that appraisers have: 



a sound knowledge and understanding of the work of each appraisee (and) are 
in a position to ensure that agreed outcomes are processed through appropriate 
institutional structures. (Lee, 1991) 

The NJC also recommends some degree of choice, and then leaves the 
decision to local schemes. The issue of choice, linked with ownership 
and commitment, applies to many other aspects of appraisal design and 
will be considered in the context of the survey findings. 



For appraisal to be effective and credible all participants must be 
adequately prepared to take part. It is generally recognised that 
appraisers need specific training, but not always that appraisees do. 
There is a strong argument for training appraisers and appraisees 
together. Set against this is the high resource implication, since there are 
likely to be at least five or ten times as many appraisees as appraisers. 
However, training of appraisees is now considered good practice 
(Anderson, 1993). The NFER evaluation report emphasises the critical 
role that training played in the pilot schemes and the NJC commentary 
states that: 

training is not cheap, but it is essential. (Lee, 1991) 

Preparation, location and documentation 

Decisions about other aspects of the process, such as preparation, 
location and documentation, must strike a balance between structure 
and flexibility which is appropriate to the culture of the organisation and 
the purposes of the scheme. 



Training 
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Adequate preparation by appraiser and appraisee is essential if 
appraisal resources are not to be wasted. The appropriate location for the 
appraisal interview is a matter of personal taste. However, having a 
degree of flexibility and involving the appraisee in the decision is likely 
to increase the sense of ownership and commitment to the scheme. 

Documentation should be designed to make a positive contribution to 
the process. A common criticism of schemes is that they are dominated 
by paperwork which is a distraction rather than a support (Lee, 1991). 



Monitoring and evaluation 

Monitoring a scheme to ensure that appraisals are taking place and 
outcomes are being implemented is a distinct process from evaluating 
the scheme in terms of benefits and disadvantages at an individual and 
organisational level. The extent and nature of the monitoring and 
evaluation which takes place is not clear from the literature. There is 
evidence of an increasing emphasis on the need to identify 
organisational as well as individual benefits (for example, Anderson, 
1993), so that appraisal fulfils the strategic role advocated, for example 
by Guest (1989). However, there is also a recognition that such 
evaluation is 'notoriously difficult to do' (Yeates, 1990). 

With respect to further education, the NJC framework (1991) 
recommends ongoing monitoring and evaluation. It suggests that this 
includes considering the effectiveness of the link between individual 
appraisal and wider review issues, reinforcing its strategic role. 

Effectiveness of appraisal 

Much of the literature identifies potential benefits and disadvantages of 
appraisal, but comprehensive information about the relative costs and 
benefits of appraisal has not emerged. One of the more comprehensive 
surveys (Long, 1986) reports on the perceived strengths and weaknesses 
of 250 appraisal schemes. 
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Chapter 2 

Organising the research 



Deciding on the approach 

Identifying the key issues in appraisal from the literature review 
revealed a framework around which the research could be structured. 
To fulfil its objectives the research would have to be conducted in such a 
way as to achieve: 

• comprehensive coverage of the sector, giving information 
about as many colleges as possible rather than detailed 
information about schemes in a small number of colleges 

• a cross-section approach which would give a comprehensive 
picture of the current situation, rather than a longitudinal 
study giving historical information about the process of 
scheme development. 

These criteria called for an approach which would enable the collection 
and analysis of a large volume of data. The data needed to be in a form 
which would facilitate structured and meaningful quantitative analysis, 
while still retaining enough detail to address the research questions, 
identified in the Executive summary. 
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Using a questionnaire 

Data would be collected via a carefully designed questionnaire, rather 
than by interviews, because it would only have been practicable to 
conduct interviews in about five per cent of colleges, a very small 
sample size. The questionnaire would be sent to 653 public sector FE 
institutions in England, Wales and Scotland. 

For much of the information required there were only a small number of 
probable responses. So to increase the probability of completion, and for 
ease of analysis, most of the questionnaire involved fixed response 
answers using tick boxes. Where other responses would be meaningful, 
text boxes were included for the respondent to record the details. 

One area for which an open-ended question was required concerned the 
identification of perceived benefits and disadvantages of the schemes. In 
this instance the response would be based on the respondent's 
judgement of the situation. Categorising the question according to the 
purposes of the scheme may have created an artificial link between 
purposes and their accomplishment. It would also have prevented any 
unanticipated benefits from being revealed. The responses to this 
question would be considered in the context of identified purposes and 
the overall aim of enhancing the institution's quality of service. 

The questionnaire was designed to address the research questions, with 
information requested on the areas outlined in Table 2. A full list of the 
questions included in the questionnaire is given in Appendix 3. 

Limitations of the methodology 



Representativeness of the sample 

Sending the questionnaires out by post would mean that there would be 
no control over the survey returns and therefore no control over the 
sample of respondents. Although in this case the response rate was 
relatively good (36%), it still means that the results include no data on 
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Extent OF implementation in FE 

• Number of schemes in the institution 

• Coverage of schemes 

• When the schemes were introduced 

Characteristics 

• Strategic context 

Status with respect to Investors in People Standard 
Purposes 

• Design and implementation 

Choice about participation 
Relationship with other procedures 
Frequency of appraisal 

Model of appraisal/identity of appraisers - others having input 

Allocation of time for appraisal 

Degree of choice of appraiser 

Appeals procedure 

Training 

Documentation 

Access to records 

Evidence used, including teaching observation and task evaluation 
Initial meeting 
Exchange of documents 
Location 

Identification of outcomes 
Levels of monitoring 
Levels of evaluation 

Effectiveness 

Perceived benefits 
Perceived disadvantages 
Cost-benefit analysis 
Planned changes 

Table 2: categories of information requested in the questionnaire 
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420 institutions. This makes it particularly important that the 
respondents constitute a representative sample with respect to size and 
type of institution. However, if the questionnaire is sent to all 
institutions there is no possibility of ensuring this. One possibility would 
be to send the questionnaire to a representative sub-section of 
institutions and follow them up actively to encourage a higher response 
rate. This option was rejected because there is still no guarantee that the 
sample would be representative, and the database would have been 
smaller. Also, even if the sample were to be representative with respect 
to size and type of institution it may not be representative with respect 
to the data requested. 

Colleges which have implemented appraisal schemes are more likely to 
return the questionnaire than colleges which have not. The results of this 
survey on the extent of implementation within the sector may therefore 
be an overestimation. 



Survey respondents 

The questionnaires were completed by one individual within each 
institution and therefore represent a particular perspective, usually that 
of a human resources or staff development professional working in a 
cross-college capacity. 

Human resource specialists may have a more positive perception of the 
appraisal scheme than other staff. The questionnaire did not collect 
systematic information about the precise role of the survey respondents, 
which is a fault of its design. 



Quantitative versus qualitative data 

The use of a tick box approach to encourage questionnaire completion 
and simplify analysis means that the data lacks richness. However, a 
survey of this size has to sacrifice quality for quantity, to some extent. 
Content analysis has been used to provide meaningful interpretation of 
some qualitative data, but this brings its own problems. The 
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identification of appropriate categories is a matter of judgement and will 
depend upon the perceptions of the individual researcher. The 
aggregation of the data will again mean loss of detail. 

It is helpful to be aware of these methodological limitations. Every effort 
has been made to address them in the design of the questionnaire and 
the analysis of the data. Particular difficulties are referred to in the 
discussion of the findings, where appropriate. 



Analysis of data 

The questionnaire responses have been analysed as follows: 

• quantitative data has been analysed using SPSS (Statistical 
Package for the Social Sciences) software, to convert frequency 
data to percentages — this provides information about the 
position of colleges with respect to Investors in People 
Standard, the number of appraisal schemes, the major 
characteristics of schemes, and the extent to which they have 
been formally monitored and evaluated 

• data in the 'other' category has been checked by reading text 
boxes to ascertain if any frequently occurring options have 
been omitted from the tick box options included in the 
questionnaire; because too few examples of any one option 
emerged, this data is not analysed in any depth but is referred 
to, as appropriate, in the discussion of findings 

• responses to the open-ended questions about perceived 
benefits and disadvantages, and about planned changes, have 
been analysed to identify recurring themes which have then 
been grouped into categories (keeping an 'an open mind' 
about those used to avoid bias in interpretation [Bell, 1993]); 
the frequencies have been converted into percentages to 
provide quantitative information on these aspects. 
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Chapter 3 

Appraisal: the emerging picture 



Out of the 653 institutions sent the questionnaire, 233 replied - a 
response rate of 36 per cent. The profile of the sample in terms of type 
and size of institution is shown in Figures 1 and 2 respectively (see 
Appendix 4). The sample provides a fairly representative cross-section 
of the total population with respect to size and type. However, it is not 
necessarily representative with respect to the implementation of 
appraisal, since colleges which have a scheme are more likely to respond 
to the questionnaire than those without a scheme. The main findings of 
the survey are discussed here in the context of the research questions. 
The data is represented in appropriate graphs in Appendix 4. 



Extent of implementation in FE 

As shown in Figure 4, a high proportion of colleges completing the 
questionnaire (94%) have introduced appraisal schemes. The 
overwhelming majority of these schemes (78%) were introduced 
between 1992 and 1996, with the major concentration of new schemes 
(57%) in 1992, 1993 and 1994 (see Figure 5); 16.9 per cent gave no 
response to this question. 
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This suggests that schemes have been implemented as a result of the 
NJC agreement, and that colleges have responded positively to the 
agreement, despite the changed circumstances and additional workload 
created by incorporation. 

Figure 6 shows that, in many cases, FE colleges have used the 
opportunity provided by the NJC agreement for appraisal of academic 
staff to introduce a single scheme for all staff which should promote 
greater understanding, a shared commitment to strategic objectives and 
positive working relationships. However, nearly 45 per cent of survey 
respondents have separate schemes for different categories of staff. 
Because the NJC appraisal framework was agreed rather than imposed 
and has taken account of a range of perspectives, and of the need for 
appraisees to have ownership, schemes for all staff have the potential to 
make a positive impact. 

Characteristics of appraisal schemes 



Strategic context 

Figure 8 shows that the two main objectives of FE appraisal schemes are 
to address staff development needs (92% of schemes) and identify 
targets linked with organisational objectives (70% of schemes). 
Respondents were asked to assign a rating from 1 (very little) to 5 (very 
much) to indicate the level to which their scheme was designed to meet 
the specified purpose. There is little evidence of schemes being designed 
to link with individual or group performance-related pay: for more than 
70 per cent of schemes this was described as non-applicable; 14 per cent 
of respondents gave no response, and 12 per cent said that their schemes 
were linked with performance-related pay to a small extent. 



This suggests that colleges see staff development and the setting of clear 
targets as more effective in improving performance than the use of 
extrinsic motivators such as performance-related pay. This fits with what 
the general and the FE-specific appraisal literature suggest is 
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appropriate for organisations whose major business is teaching and the 
promotion of learning. For example, Long (1986) found in his survey of 
250 schemes that: 

Despite assumptions to the contrary, there is no evidence to show a growth in the 

use of performance review for the assessment of salary increases and merit awards. 

More recently, Fletcher (1993), despite reporting Bevan and Thompson's 
(1991) finding that performance-related pay has increased 'in recent 
years', states that: 

both research and experience indicate that this (ie performance-related pay) 

can create many pitfalls in constructing an effective appraisal scheme. 

The relationship between the specific purposes of appraisal and its 
overall aim of improving institutional quality needs to be considered in 
the context of the extent to which it contributes to the achievement of the 
college's strategic objectives. If appraisal schemes with the explicit 
purposes of staff development and meeting targets are also to enhance 
quality of service they must: 

• carry out the review of individual performance, the 
identification of staff development needs, and the setting of 
targets linked to organisational objectives in the context of 
improving quality of service 

• take into account the perspectives of students and other 
college clients with respect to the quality of service 

• make links with other quality review mechanisms such as 
course/ programme review, section /departmental review and 
institutional review, including college inspection 

• have effective mechanisms for linking appraisal outcomes 
(whether staff development outcomes or targets linked with 
organisational objectives) with, and therefore ensuring they 
contribute to, organisational performance. 
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The position of colleges in the sample with respect to the Investors in 
People Standard gives some indication of the context in which appraisal 
is undertaken, since the Standard requires institutions to review staff 
development needs in the context of business objectives. 

Figure 3 shows that 65 per cent of participating colleges have committed 
themselves to the Standard, and 30 per cent have obtained it. This 
suggests an understanding of the need to review staff development 
needs in the context of strategic objectives, although it provides more 
evidence of intention than of achievement. 

In the context of the strategic role of appraisal in enhancing institutional 
quality, the actual achieving of the Investors in People Standard is an 
important factor. As Smith (1995) states: 



the Standard, if approached sensibly, is an excellent tool for ensuring that 
human resource tactics deliver an organisation's strategy more effectively. 

The extent to which contribution to organisational performance is iden- 
tified as a benefit of staff appraisal sheds further light on the role 
appraisal is playing at a strategic level. Table 7 (see page 45) shows that 
42 per cent of schemes identified this as a benefit. A range of other ben- 
efits for organisational performance are also identified. This is encour- 
aging, although Table 3 shows that the overall number of statements 
about benefits for organisational performance is substantially lower than 

the number of statements about 
benefits for the appraisers or for 
appraisees. However, these are the 
perceptions of survey respon- 
dents only. Figure 27 shows that 
only 20 per cent of schemes have 
been formally evaluated to 
identify benefits for organisa- 
tional performance. This is 
probably more due to the difficult 
nature of carrying out such evalu- 



Type of benefit 


Number of 
statements 


Benefits to appraisee 


631 


Benefits to appraiser 


483 


Benefits to team 


263 


Benefits to organisation 


337 



TABLE 3; NUMBER OF STATEMENTS 
ABOUT DIFFERENT TYPES OF BENEFITS 
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ation (as revealed by the literature review) than a lack of interest in 
organisational benefits. Despite this, consideration of Table 4 suggests 
that the potential of appraisal outcomes to make a strategic contribution 
to organisational performance is not yet fully achieved. 



Design, implementation and evaluation 

A scheme is more likely to contribute to staff development, the 
achievement of targets linked with organisational objectives and the 
overall aim of enhancing the service if appraisal is carried out in such a 
way as to be motivating, credible and effective. The attitudes of 
appraisers and appraisees towards appraisal will have a direct impact 
on the effectiveness of schemes (Anderson and Barnett, 1987). 

The 1991 NJC framework for FE appraisal schemes was carefully 
developed to achieve the aims of motivation, credibility and 
effectiveness. It reflects good practice with respect to the design, 
implementation and evaluation of appraisal schemes, identified in the 
literature. It is therefore helpful to consider the findings of this survey in 
the context of the relevant NJC framework recommendations. Table 4 
provides a profile of the survey results against the NJC framework. 

Models of appraisal 

Appraisals are overwhelmingly (82%) carried out by line managers or 
other managers. Only two per cent of schemes are 360° appraisal models 
which involve a range of perspectives (Figure 12 gives the details). 

The extent to which customers have an input is important. Figure 13b 
shows that this does happen to some extent, with students having an 
input to 15 per cent of schemes, and other college clients having an input 
to seven per cent of schemes. However, for many schemes none of these 
groups have an input. The majority of additional inputs come from 
managers (in 39% of schemes) and peers (in 27% of schemes), rather than 
from customers. 
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Table 4: Profile of survey results against NJC framework 



Characteristic 


NJC recommendation 


Survey findings 


Comment 


Participation 


Participation required for all 


Participation 


Survey findings 


in scheme 


full-time academic staff and 


compulsory in 


generally in line 




all part-time academic staff 


83% of schemes, 


with NJC 




on proportional contracts 


voluntary in 4% 
and compulsory 
for some staff in 
6% of schemes 


framework 


Identity of 


To be determined by 


Line manager 


Substantial 


appraiser 


governing body; to meet 


is appraiser in 


evidence of 




specific criteria: sound 


68% of schemes; 


line manage- 




knowledge and under- 


other manager 


ment model, 




standing of appraisee's 


in 14% of 


which has the 




work; knowledge of proce- 


schemes; peer 


potential to 




dures and developments 


appraisal in 8% 


meet identified 




within each institution; be 
in a position to ensure 
agreed outcomes are 
processed 


of schemes 


criteria 


Choice of 


Appraisees should be offered 


Constrained 


Some choice. 


appraiser 


an alternative choice 


choice in 52% 


so combines 




of appraiser 


of schemes, free 


strategic link 






choice in 4% of 


with a degree 






schemes, and no 


of flexibility; 






choice in 27% of 


however. 






schemes. 


schemes with 






Schemes with no 


no choice may 






choice do not 


be indirectly 






meet NJC 
agreement 


discriminatory 
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Characteristic 


NJC recommendation 


Survey findings 


Comment 


Overall 
aim and 
purposes; 
strategic 
context 


Schemes should aim to 
enhance the quality of the 
service provided by the 
institution; professional 
development of staff to be 
seen in the context of 
institutional and unit 
development plans; links 
with institutional and 
departmental review 
essential 


Primary purposes 
of schemes are 
staff development 
and targets linked 
with organisational 
objectives; major 
benefits include 
staff development, 
communication 
and (although 
more limited) 
contribution to 
organisational 
objectives 


Purposes and 
identified 
benefits of 
schemes 
suggest they 
make a 
significant 
contribution to 
meeting overall 
aim, although 
more at an 
individual than 
an organisational 
level 


Relationship 
with other 
procedures 


Schemes should be 
complementary to, but 
must be separate from, 
disciplinary, probation, pay 
and promotion procedures 


Strong evidence 
of complete 
separation from 
the following 
procedures; 
disciplinary, 
grievance, 
capability, pay, 
promotion and 
references 


Survey findings 
suggest that 
good practice 
laid down in 
NJC agreement 
is closely 
followed 


Teaching 

observation 


Evaluation of individual's 
effectiveness in the 
facilitation of learning 
should be included. 
Decision about class- 
room observation left 
to local schemes 


Teaching 
observation 
included in 64% 
of schemes. Task 
evaluation 
included in 54% 
of schemes 


Findings reflect 
NJC framework 
and demonstrate 
a significant 
commitment of 
resources to a 
thorough 
approach to 
appraisal 
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Characteristic 


N)C recommendation 


Survey findings 


Comment 


Other 

evidence 


Relevant data should 
include information from 
college and unit reviews; 
job descriptions; teaching 
and/or task evaluation 
records; previous 
appraisal records 


Evidence used: 
review of previous 
appraisal 70%; 
section reviews 
30%; inspection 
feedback 29%; 
and course 
reviews 9% 


Findings reflect 
NJC framework 


Training 


Access to training 
required for all 
participants. No 
recommendation about 
whether training for 
appraisers and 
appraisees should be 
joint or separate 


Training provided 
in 92% of 
schemes; 
appraisers pro- 
vided with more 
training on 
average than 
appraisees. 
Appraisers and 
appraisees trained 
together in 29% of 
schemes, and 
separately in 43% 
of schemes (some 
used both 
approaches) 


High level of 
training 
provided; the 
survey findings 
reflect, in 
general, the 
good practice 
in the NJC 
framework 


Self-appraisal 


Completion by appraisee 
of locally standardised 
self-appraisal exercise 


90% of schemes 
include self- 
appraisal form 
for appraisee 


Self-appraisal 
makes a 
significant 
contribution to 
the process, as 
required by the 
framework 
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Characteristic 


NJC recommendation 


Survey findings 


Comment 


Outcomes 


An agreed record 
focusing on 
outcomes and 
objectives is 
required 


Staff development out- 
comes are identified for 
93% of schemes; other 
objectives are identified 
for 8i% of schemes. Full 
details of outcomes are 
recorded less frequently 


There is a clear 
understanding of 
the need to record 
outcomes; 
however, the 
full intentions 
of the framework 
are not always 
achieved in 
this respect 


Monitoring 
and eval- 
uation 


Arrangements for 
monitoring and 
regular evaluation 
are required to be 
made locally, with 
respect to number 
of detailed aspects 
of appraisal 


Monitoring to ensure that 
appraisal is taking place 
occurs in 96% of schemes. 
Monitoring of outcome 
identification, implement- 
tation and review takes 
place in 67%, 52% and 
50% of schemes respect- 
ively. Evaluation to 
identify benefits at 
different levels has 
taken place as follows: 
appraisees 40%, 
appraisers 35%, group/ 
team performance 7% 
and organisational 
performance 14%. 
Cost-benefit analysis 
has occurred for 7% of 
schemes 


Although some 
monitoring of 
appraisal is 
taking place 
this is 

happening to a 
lesser degree 
and in a less 
detailed way 
than envisaged 
by the 
framework 
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Characteristic 


NJC recommendation 


Survey findings 


Comment 


Frequency 


Framework requires 
regular appraisal, at 
least every two 
years, but 
recommends more 
frequent appraisal 
if resources 
are available 


Appraisal is annual for 
48% of schemes and 
every two years for 
41% of schemes 


An encouraging 
number of 
schemes are 
annual. Frame- 
work require- 
ments and recom- 
mendations are 
both reflected in 
findings 


Complaints/ 

appeals 


Local procedure 
required for hearing 
and disposing of any 
complaint about the 
appraisal process or 
conclusions 


76% of schemes have 
appeals procedures; 
procedures are known 
to have been used in 
only 10% of schemes. 
15% of schemes report 
no appeals procedure 


In general, survey 
findings reflect 
NJC framework, 
although the 
number of 
schemes which 
do not meet the 
requirement to 
have an appeals 
procedure is 
surprisingly high 


Initial 

meeting 


Initial meeting 
between appraiser 
and appraisee 
is required unless 
both agree it is 
unnecessary 


87% of schemes have 
provision for a pre- 
meeting, which takes 
place in the majority 
of cases 


Survey findings 
reflect the NJC 
framework 


Access to 
appraisal 
reports 


Agreed appraisal 
record kept in 
appraisee’s confiden- 
tial personnel record; 
additional copies 
available to appraisee 
and appraiser 


Access to full appraisal 
records is in general 
confined to appraiser 
and appraisee, 
although in a minority 
of cases principal and 
personnel managers 
have access 


Survey findings 
reflect the high 
level of confiden- 
tiality in framework 
requirements 
and demonstrate 
awareness of the 
sensitivity of this 
issue 
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Link with other quality review mechanisms 

There is strong evidence that appraisal is informed by targets identified 
in previous appraisal rounds (in 70% of schemes). Figures 21a and 21b 
give the details. This suggests a continuous quality improvement 
approach, building on previous achievements. There is evidence of a 
link between the review of performance at an individual level and 
section reviews (occurs in 30% of schemes). Course reviews contribute in 
nine per cent of schemes and institutional reviews (such as college 
inspections) in 29 per cent of schemes. Appraisal is also shown to be 
informed by the observation of teaching (for 64% of schemes) and the 
evaluation of other tasks carried out by appraisees (for 54% of schemes) 
- see Figure 20. 

All of these sources of evidence can make a valuable contribution to 
performance review. The NJC pilot schemes revealed that teaching 
observation is an important learning experience for observers and those 
observed, but is also resource intensive (Lee, 1991). It is encouraging that 
the majority of schemes in the survey involve teaching observation as 
part of the review process, despite the high resource implications. 

Design and implementation 

The findings of this research suggest that the FE appraisal schemes do, 
in practice, reflect closely the NJC framework in many aspects of design 
and implementation (see Table 4). Given the high degree of good 
practice reflected in the NJC framework, this result is encouraging. It 
means that appraisal schemes in further education have been designed 
and implemented to play a strategic role by combining the development 
of the individual with a contribution to organisational performance. It 
suggests that: 

• appraisees have taken an appropriate degree of ownership of 
the process 

• the appropriate balance of structure and flexibility has been 
generally achieved 
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• the development and implementation of schemes has taken 
account of staff development and business needs in the 
context of the FE culture. 

The two main aspects which do not reflect recommended good practice 
are ones which are more difficult to deal with, namely: 

• setting appraisal schemes in the wider context of unit and 
institutional review 

• the extent and level of detail of monitoring and evaluation. 

However, the survey does reveal a high level of awareness of the need 
for development in these areas, and evidence of development in 
progress (see Tables 10 and 11 on page 49). 

Effective mechanisms 

The extent to which appraisal outcomes of various types are identified, 
recorded and monitored is shown in Figure 24. This data suggests that a 
high percentage (70-90%) of schemes involve identifying and recording 
specific outcomes. There is some drop-off between identification and 
recording and between recording and monitoring, but it is not great. 
Figure 25 shows the different details recorded for the two main out- 
comes, those linked to staff development needs and those linked to 
organisational targets. With respect to staff development and targets 
linked to organisational objectives, the action to be taken is recorded 
more frequently (78-91% of schemes) than the person(s) responsible (in 
approximately 70-80% of schemes) or the timescale (in approximately 
70-80% of schemes). This information is recorded more frequently than 
a review mechanism for an identified outcome (in approximately 50-60% 
of schemes). The cost of implementing these outcomes is only recorded 
in approximately 20 per cent of schemes. This finding is compatible with 
Anderson's (1993) report in which he states that: 

even in well organised and well run appraisal systems only a proportion 
of actions planned and agreed by appraisers and appraisees subsequently 
are implemented. 
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Evaluation 



Approximately 40 per cent of schemes evaluate benefits for the 
appraisee, and almost as many evaluate benefits for the appraiser 
(35% of schemes). There is a much lower level of evaluation at the 
organisational level (in only 14% of schemes). Figure 27 gives the details. 
This suggests that the focus of evaluation is at the individual level, and 
that appraisal is not always being considered in the context of the overall 
aim of enhancing institutional quality. 



Effectiveness of appraisal 

Analysis of statements about the perceived benefits and disadvantages 
of appraisal reinforces the emphasis that these schemes place on the 
individual. Table 3 shows that there are far more statements about 
benefits identified for the appraisee and appraiser than for the team or 
the organisation as a whole. The nature of the benefits are more strongly 
focused on the purposes of staff development. However, targets linked 
with organisational objectives are also frequently identified as a benefit, 
and there is some evidence of the strategic role of appraisal in benefiting 
organisational performance. 



Benefits for the appraisee 

The nature of benefits identified for appraisees (see Table 5) reflects 
strongly the two major purposes of the scheme: meeting staff 
development needs and identifying targets linked with organisational 
objectives. These findings provide strong evidence that appraisal 
schemes are meeting their identified purposes (particularly since they 
are the result of content analysis on open-ended questions, rather than 
of the selection of pre-identified options). 

The second most important benefit identified is the opportunity for 
communication between appraiser and appraisee (usually the 
appraisee's line manager). This emphasises the importance of investing 
in people in order to enhance the quality of their contribution. It is an 
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Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Staff development 


An opportunity to identify staff 
development needs and how 
they are to be met 


155 


57 


Communication/ 

relationships 


An. opportunity for communication 
with the appraisee’s line manager 
providing an in-depth dialogue and 
exchange of views, attention and 
quality time 


109 


40 


Objectives linked 
to work role 


Identification of individual 
work- related objectives 


73 


27 


Performance review 
and improvement 


Review of performance, leading 
to improved performance 


55 


20 


Role 


Review and clarification of role 


46 


17 


Recognition 


Recognition of achievements 
of appraisee 


36 


13 


Motivation 


Improved morale, a greater sense 
of value and belonging, and 
increased motivation and 
commitment 


36 


13 


Influence 


Opportunity to influence 
developments and contribute to 
planning 


35 


13 


Career 

development 


Opportunity to identify and 
communicate career aspirations 


28 


10 


Resources/ 

support 


Access to support/resources/ 
improved working arrangements 


25 


9 


Feedback 


Feedback on performance from the 
line manager (appraiser) 


23 


8 


Relationships 


Building relationships between line 
manager (appraiser) and appraisee 


15 


6 


Organisational 

objectives 


Greater awareness of 
organisational objectives 


10 


4 



Table 5: Perceived benefits of appraisal for appraisees 
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important factor to take into account when identifying quantifiable and 
non-quantifiable benefits for cost-benefit analysis of appraisal schemes. 
The range of appraisee benefits identified in Table 5 reflects those 
identified by Anderson (1993). They show that appraisal schemes in FE 
have succeeded in blending developmental and evaluative aspects of 
appraisal so that they complement rather than conflict with each other. 

Benefits for the appraiser 

The most striking features of benefits identified for appraisers (see Table 
6) are the extent to which appraisal increases their awareness of the 
content of their staff's jobs and the quality of their performance. The 
other main benefit identified is the contribution appraisal makes to the 
appraiser's development of management and communication skills. 
Given the relatively low levels of investment in management 
development in the UK, this result is not surprising. This benefit should 
certainly be considered when conducting cost-benefit analysis to 
identify the extent to which appraisal is a worthwhile investment. 



Benefits for organisational performance 

Although there were fewer statements identifying benefits for 
organisational performance (see Table 7), the contribution of individual 
objectives to the meeting of organisational objectives emerged strongly 
as a perceived benefit for 42 per cent of schemes. This indicates that 
appraisal schemes do play some role at a strategic level, as Anderson 
(1993) and others have suggested that they should. However, the 
relatively small number of references to the contribution of appraisal to 
the achieving of the Investors in People Standard (seven statements) - 
with its focus on a strategic approach to staff appraisal - and to 
improving the learning experience for students (22 statements), 
reinforces the finding that appraisal outcomes have had more impact at 
the individual than at the organisational level. 
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Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Appraiser aware- 
ness of issues 


Greater awareness of issues which 
concern and interest staff, and of 
their priorities and aspirations 


155 


57 


Appraiser aware- 
ness of jobs 


Greater awareness of the content of 
staff jobs and more information 
about their section of the 
organisation 


136 


50 


Appraiser awareness 
of performance 


Greater awareness of staff 
performance 


78 


29 


Appraiser 

development 


Development of the appraiser’s 
communication and management 
skills 


53 


19 


Staff development 


Opportunity to identify staff 
development needs more effectively 


47 


17 


Planning 


More informed planning 


41 


15 


Communications/ 

relationships 


Better communication with 
appraisees, the opportunity for 
dialogue; building relationships 
between line manager and appraisee 


61 


22 


Recognition 


Opportunity to recognise staff 
achievements 


36 


13 


Individual 

objectives 


Opportunity to agree individual 
work-related objectives 


25 


9 


Relationships 


Improved management/staff 
relationships 


23 


8 


Feedback 


Opportunity to review performance 
and provide feedback to staff 


23 


8 


Roles 


Review and clarification of 
appraisee roles 


23 


8 


Team development 


More effective teams 


15 


6 


Organisational 

objectives 


Communication of college 
objectives and priorities 


11 


4 


Performance 


Improved staff performance 


8 


3 


Motivation 


More motivated staff 


4 


1 



Table 6: Perceived benefits of appraisal for appraisers 
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Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Organisational 

objectives 


Individual objectives contribute to 
meeting of organisational objectives 


113 


42 


Staff 

development 


Staff development needs inform the 
strategic plan and link to 
organisational objectives 


42 


15 


Planning 


More informed and more effective 
deployment of staff and resources 


38 


14 


Motivation 


Increased motivation, sense of 
belonging and commitment 


29 


11 


Strategic link 


Effective appraisal contributes to 
meeting strategic objectives 


29 


11 


Student experience/ 
quality of service 


Improved learning experience for 
students 


22 


8 


Communication 


Improved communication 
throughout organisation 


14 


5 


Improved 

performance 


Improved individual performance 
contributes to organisational 
performance 


13 


5 


Culture 


Development of a more open 
culture, partly as a result of a 
single scheme for all staff 


10 


4 


Team performance 


Improved team performance 


7 


3 


Monitoring 


Mechanism for monitoring progress 
against targets 


7 


3 


Investors in People 


Contributes to the achieving of the 
Investors In People Standard 


7 


3 


Productivity 


Greater productivity 


6 


2 



Table 7: Perceived benefits of appraisal for organisations 
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Benefits for teams 



Hartle (1995) refers to team appraisal as having significant potential in 
contributing to staff motivation and organisational performance. 
Although there are fewer statements identifying benefits from appraisal 
for teams (263 in total - see Table 8), they do provide some evidence of 
the positive impact within the organisation of appraisal at this level. 
Interestingly, there are more statements identifying motivation as an 
appraisal benefit for teams (43) than as a benefit for individual 
appraisees (36), appraisers (4) or organisations (29). There is room for 
further development which builds on the potentially motivating effect of 
a team approach to appraisal. This development would fit well with the 
focus on the establishment of self-imp roving teams as part of the self- 
assessment process (see, for example, Dixon, 1996). 



Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Team objectives 


Team objectives are identified in the 
context of organisational objectives: 
individual objectives complement 
team objectives 


51 


19 


Improved 

performance 


Increased effectiveness of team 
performance 


50 


18 


Motivation 


Greater team spirit and commitment 
to shared goals 


43 


i6 


Awareness of 
performance 


Teams have a greater awareness of 
their performance and evaluate their 
activities more 


32 


12 


Staff development 


Team development needs are 
identified and addressed 


27 


10 


Communication 


Improved communication within and 
between teams 


24 


9 


Planning 


More effective planning of team activities 


18 


7 


Strategic link 


Team performance contributes more 
effectively to meeting strategic objectives 


11 


4 


Student experience/ 
quality of service 


Better quality of service for students and 
improved learning experience 


7 


3 



Table 8: Perceived benefits of appraisal for teams 
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Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Time 


Appraisal makes very heavy demands 
on staff time, particularly for appraisers 
and those responsible for managing 
and administering the schemes 


79 


29 


Lack of 
commitment 


Lack of interest, no real commitment 
or actual hostility 


57 


21 


No strategic 
link 


Information from appraisal is not linked 
with other organisational procedures 


44 


i6 


Evaluation 


Schemes are difficult to evaluate 


29 


11 


Unfulfilled 

expectations 


Schemes raise expectations by 
promising outcomes which are not 
implemented, causing demotivation 
and lack of credibility 


28 


10 


Bureaucracy 


Schemes are bureaucratic - unwieldy, 
awkward and cumbersome 


i6 


6 


Variable 
appraiser skills 


Schemes rely heavily for their quality 
on the skills of appraisers, which 
vary considerably 


15 


6 


Workload 


Schemes make unreasonably high 
demands on workload, particularly 
for some appraisers 


12 


4 


Paper- 

dominated 


An excess of forms and other 
documentation 


9 


3 


Too infrequent 


Biennial schemes found not to be very 
satisfactory 


9 


3 


Model 


Schemes are perceived to be too 
‘soft’, not judgemental enough and 
giving staff too much choice 


6 


2 


Inappropriate 

coverage 


Coverage of scheme not appropriate 


6 


2 


Stress 


Additional workload is stressful 


5 


2 



Table 9: Perceived disadvantages of appraisal 
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Disadvantages of appraisal 

Colleges see the heavy time demand that appraisal puts on staff as the 
main disadvantage - this was the most frequent response with 79 
statements. Table 9 shows the other responses made. None of the 
planned changes identified (see Table 10) address the time demand issue 
directly, although there is evidence of plans to simplify appraisal 
schemes (the most frequently mentioned planned change with 27 
statements). Given the perceived contribution to appraisers' 
development, the time invested in appraisal would seem to be justified, 
and should perhaps be more properly allowed for. The perceived lack of 
commitment to schemes (57 statements) may be partly a result of severe 
time pressures as well as other factors. Continued improvements to 
schemes and ongoing development of appraiser skills should in the long 
term help to reduce the time demands. The other most frequently 
mentioned disadvantages are the lack of a link between appraisal and 
other organisational procedures (44 statements), and the difficulty of 
evaluating appraisal (29 statements), both of which have already been 
discussed. The changes that survey participants plan to effect (as 
indicated in Table 10) show that the disadvantages identified with 
respect to bureaucracy, frequency, purpose and coverage of the appraisal 
scheme are being addressed. 

Cost-benefit analysis 

Figure 28 indicates that only seven per cent of schemes have carried out 
cost-benefit analysis. A review of the comments made by respondents on 
this issue suggests that many institutions perceive cost-benefit analysis 
in simple terms only, interpreting the question as meaning actual cost of 
the scheme, or looking at it from the point of view of the number of days 
middle managers have to commit to appraisal, without analysing the 
cost. There is no evidence of comprehensive cost-benefit analysis involv- 
ing the identification of quantifiable benefits and costs, but this could be 
due to colleges misinterpreting the question rather than providing an 
accurate picture as to how many colleges carry out such an analysis. 
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Category 


Benefit 


Number of 
statements 


Percentage 
of schemes 


Simplify scheme 


Simplify scheme procedures 
and/or simplify documentation 


27 


10 


Strategic link 


Link appraisal scheme more closely 
with other quality approval systems 
and organisational planning 
processes 


27 


10 


Co-ordination, 
monitoring and 
evaluation 


Improve management of the system, 
particularly with respect to 
monitoring and evaluation 


26 


10 


Refocus 

purpose 


Focus more on objective setting in 
addition to staff development 


25 


9 


Frequency 


Move to annual appraisal, rather 
than biennial 


20 


7 


Broaden model 


Broaden the sources of information 
used to inform appraisal, including 
the introduction of teaching 
observation, 360“ appraisal, upward 
appraisal and peer appraisal 


20 


7 


Extend coverage 


Extend coverage of scheme from 
full-time academic staff to all staff 


16 


6 


Move to single 
scheme 


Integrate existing schemes for 
academic and support staff into a 
single scheme for all staff 


11 


4 


Line manage- 
ment model 


Make greater use of line managers 
as appraisers 


9 


3 


Improve 

training 


Improve training to prepare staff 
for appraisal 


5 


2 



Table 10 : Nature of planned changes to appraisal schemes 





Number of statements 


Appraisal review has already taken place 


9 


Appraisal review is currently taking place 


7 


Appraisal review is planned 


23 


Appraisal review is ongoing, as are changes 
to the scheme 


18 



Table 11 : Extent of evaluation of appraisal schemes 
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Chapter 4 

Implications of findings 



Results of this survey offer an encouraging picture of widespread 
successful implementation of appraisal in the FE sector. That this has 
been achieved over a relatively short period in the early 1990s, following 
significant difficulties in the latter half of the 1980s, is a powerful 
endorsement of the role of joint negotiation, and the value of taking 
principles of good practice seriously, via the introduction of the NJC 
framework. Table 4 summarises the research findings in the context of 
this framework, revealing the extent to which colleges have adopted the 
aspects of good practice put forward. 



Findings testify to the positive contribution appraisal can make to 
individual and organisational performance. The relatively high survey 
response rate is in itself an indication of interest in the ongoing 
development and use of appraisal within the sector. However, there is 
more evidence of this potential being realised at the individual than at 
the organisational level. The strategic role that appraisal can play in 
enhancing the quality of service provided by FE institutions could be 
developed further. Appraisal needs to be informed by a wider range of 
sources and there should be more systematic evaluation of the 
contribution it can make to organisational performance. 
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The findings reinforce the view of Yeates (1990) that it is 'notoriously 
difficult' to evaluate the contribution appraisal can make to 
organisational performance. The awareness that at least some survey 
respondents have of the need for such an evaluation (see Table 11) is 
encouraging, given that such awareness is low in all sectors. It provides 
indirect evidence of the beneficial effect of the strategic role given to 
appraisal by the introduction of the Investors in People Standard. 

This difficulty with evaluation points to the need for more research into 
what aspects of the design of an appraisal scheme allow it to make a 
contribution at a strategic level. At the same time, the move within the 
FE sector to achieve continuous self-improvement and develop the 
overall quality of the college, should make evaluation of staff appraisal 
schemes a more straightforward process. 

The introduction of self-assessment within the FEFC inspection process 
(see FEFC circulars 97/12 and 97/13) provides the means to make a link 
between quality at the individual staff performance level and quality of 
the performance of the institution as a whole. 

Colleges already using teaching observation as an element of their 
performance review are in a good position to make this link, and 
according to the survey findings this is the case for many FE institutions. 
Colleges without this in place will find it difficult to undertake self- 
assessment of teaching and learning. 

Since the process is expensive, colleges will need to consider whether 
they can use one observation framework for the purposes of appraisal 
and self-assessment. Given the care that has been taken to maintain 
confidentiality of records of teaching observation, as revealed by this 
research, it is vital that this issue is approached sensitively. Colleges 
would benefit from involving relevant professional associations. In this 
way they will be able to repeat the good practice adopted by the NJC 
framework at a national level (NATFHE, 1991; page 33) when 
negotiating the appraisal framework for academic staff. 
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The research has revealed that one of the main disadvantages of 
appraisal perceived by colleges is the immense time demands it makes 
on staff, who are already dealing with stressful workloads, and that the 
time intensiveness of the process is a main cause of lack of commitment 
to schemes. 

In the absence of a high level of cost-benefit analysis, additional time 
allocation for appraisal may be difficult to justify. The positive results 
that appraisal brings may be at the cost of unreasonably high workloads 
for some appraisers. This factor needs to be considered in the context of 
the human and economic consequences of high stress levels and the 
employers' responsibilities in this area. 

Colleges need to decide how seriously they want to take appraisal. The 
many benefits it offers, both at an individual staff and at an 
organisational level, as revealed by this research, would seem to justify 
the time that needs to be invested. A cost-benefit analysis would provide 
colleges with critical information about the potential value that a well- 
resourced appraisal scheme would bring. 

Carrying out such an analysis would also contribute to the drive to 
adopt a more business-like approach to education. Once the analysis has 
been made, one desirable approach would then be to incorporate time 
for appraisal into forward planning. 

To ensure that the process is as effective as possible, it is vital that all 
taking part are prepared for what is involved. The literature review 
points to the many benefits that training appraisees and appraisers 
together brings. The staff prefer it, since it establishes a relationship of 
trust between appraiser and appraisee. In particular it demystifies the 
process for appraisees, since it makes it more transparent, allowing all 
participants to be clear about what is involved. It also makes it more 
likely that appraisees will go into the process viewing it in a positive 
light, because they see it as a means to improve their objectives and to 
influence organisational change for the better. 
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However, this research has shown that nearly half of the participating 
colleges train staff separately. Colleges tend to give appraisers two days 
of training, while appraisees are often briefed over a lunchtime meeting. 
Since there are up to ten times as many appraisees as appraisers, to give 
appraisees the same amount of training would greatly increase costs. 

However, it is desirable to move towards a model of joint training. 
Colleges with a scheme in place will now be in a better position to 
achieve this, because new staff coming in will be in much smaller 
numbers, so it will now be more cost-effective to train them together. 

The findings show that most colleges are not complacent about their 
existing schemes - just a quarter said they had no plans to change their 
scheme (see Figure 31). This commitment towards bettering their 
appraisal process could also be seen as evidence that colleges are taking 
seriously their responsibility towards self-improvement. 

Interestingly, the most frequently plarmed changes were to simplify the 
scheme and to improve the links with organisational plarming, which is 
further evidence that colleges want to improve the strategic role that 
appraisal can take, as discussed above. 



While the survey did not ask colleges for details on how they plarmed to 
effect these changes, observations of schemes subsequent to this survey 
have indicated that a basic way to simplify a scheme is to reduce the 
recording processes involved. Many colleges provide a full record of the 
appraisal interview, as well as of the action plans set. The transcribing 
involved makes this a time-costly process. However, some colleges just 
work with action plans without any detriment to the scheme. More 
colleges could adopt this practice to make their appraisal scheme 
simpler, at the same time achieving savings in terms of cost and time. 



All evidence shows that it is good practice to keep appraisal separate 
from disciplinary, grievance, capability, promotion and references 
procedures. The NJC framework recommends that this is the best 
approach to take. The findings of this research indicate that many 
colleges have taken this on board when designing their schemes. For 
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example, more than 70 per cent said that their appraisal scheme was not 
linked with performance-related pay. This is in line with other 
developments with respect to performance management in general and 
offers a useful message to colleges about to design an appraisal scheme. 

Evidence from the literature review stresses the value of a 360" model 
which involves evidence from many sources including students, other 
college clients, and from course reviews and college inspection. These 
researchers pointed to the potential this model offers in ensuring that 
employees meet their performance objectives. 

However, evidence from this research shows that in practice this model 
is very rarely used (in only 2% of cases). Fewer than 30 per cent of 
schemes used evidence from section reviews and feedback from 
inspection to contribute to appraisal, and only nine per cent involve 
evidence from course reviews. 

It is likely that colleges have decided that the 360" model makes the 
appraisal process far more complicated than perhaps is necessary and is 
too costly to run. The model which emerged as the most popular one 
was that which involved the line manager carrying out the appraisal. 
This model of appraisal offers the potential to facilitate a strategic link 
and meet the criteria identified as good practice by the NJC. 
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Summary 



Implications for appraisal in general 

• Findings reinforce the value of appraisal and its potential for 
making a positive contribution to individual and 
organisational performance, but show that the latter is 
difficult to evaluate. 

• There is evidence of the beneficial effect of the strategic role 
given to appraisal by the introduction of the Investors in 
People Standard. 

• Findings endorse the value of joint negotiation and of taking 
principles of good practice seriously. 

Implications for appraisal in the FE sector 

• The strategic role that appraisal can have in enhancing FE 
institutions' quality of service could be further developed. 

• There is a need for more systematic evaluation of the contri- 
bution appraisal can make to organisational performance. 
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Appendix 3 

Survey questionnaire 



The questionnaire asked the following questions: 

1 Please indicate the college numbers in the following categories: 

• full-time students 

• part-time students 

• full-time staff 

• part-time staff 

(data presented is for numbers of full time staff) 

2 Please indicate the college's position with respect to the Investors in 
People award: 

• have not considered submitting for the award 

• have considered submitting for the award but decided against it 

• have committed the college to seeking the award 

• have the award 



3 How many appraisal schemes for staff are there at the college? 

• None 

• One 

• Two 

• More than two 
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4 When was the scheme introduced? 

5 Who is covered by this scheme? 

6 Is the scheme compulsory or voluntary? 

7 To what extent is the scheme designed to meet the following 
purposes? 

• Staff development 

• Targets linked with organisational objectives 

• Individual performance-related pay 

• Group /team performance-related pay 

• Other 

(Please rate, then specify) 

8 Is the scheme separate from the following procedures? 

• Disciplinary 

• Grievance 

• Capability 

• Pay 

• Promotion 

• References 

9 How often are staff appraised? 

• More than once a year 

• Once a year 

• Every two years 

• Other 

10 Is there an interim review? If so how often is it carried out? 

11 In terms of who carries out individual appraisals, what models does 
the college use? 

• Line manager 

• Other manager 

• Peer appraisal 

• Upward appraisal 

• 360 appraisal 

• Other (please specify) 
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12 Who else has input to individuals' appraisals? 

• Students 

• Subordinates 

• Peers 

• Managers 

• College clients (eg employers) 

• Others (please specify) 

13 Is there any specific time allocation within staff workloads for 
carrying out appraisal responsibilities? Please specify number of 
hours for management staff, and for other staff. 

14 Is there any choice of appraiser? If yes, is it a: 

• free choice 

• constrained choice 

15 Is there an appeals procedure? If yes, has it been used? 

16 What appraisal training is provided for appraisers? and 

17 What appraisal training is provided for appraisees? 



• None 

• Half-day 

• 1 day 

• 2 days 

• Other 

Is it optional/compulsory? 

18 If training is provided for appraisers and appraisees, are they trained 
together or separately? 

19 What forms are available for use? 

• Preparation form for appraisee 

• Preparation form for appraiser 

• Full interview recording 

• Form for recording all outcomes 

• Form for recording just staff development outcomes 

• Other (please specify) 

20 Who has access to which appraisal forms? 
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21 Does the scheme include teaching observation/evaluation activities? 

22 Does the scheme include task evaluation activities? 

23 What other evidence is used for appraisal? 

• Section reviews 

• Course reviews 

• Previous targets 

• Previous appraisal records 

• Inspection feedback 

24 Is a pre-meeting held? and 

25 Are appraisal documents exchanged in advance of the appraisal 
interview? 

• Always 

• Normally 

• Occasionally 

• Never 

26 What sort of location is used for appraisal interviews? 

• Appraiser's office 

• Appraisee's office 

• Specific rooms 

• Other (please specify) 

27 Is the scheme designed to include the identificarion, recording and 
monitoring of specific outcomes? 

• Staff development 

• Targets linked with organisational objectives 

• Individual performance-related pay 

• Group/ team performance-related pay 

• Other (please specify) 

28 Are the following details recorded when outcomes are identified? 

• Action to be taken 

• Person(s) responsible 

• Timescale 

• Cost of implementation 

• Review mechanism 
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29 Is the scheme monitored in any of the following ways (and if so how 
often, by whom, and what action results)? 

• To check if appraisal is taking place 

• To check if outcomes are identified 

• To check if outcomes are reviewed 

• To check if outcomes are implemented 

30 Has there been any cost-benefit analysis of the scheme? If yes, what 
are the results? 

31 Has the scheme been evaluated in any other way to identify benefits 
for: 

• Individual appraisees? 

• Individual appraisers? 

• Group /team performance? 

• Organisational performance? 

32 To what extent do you consider the appraisal scheme benefits the 
following: 

• Individual appraisees? 

• Individual appraisers? 

• Group/ team performance? 

• Organisational performance? 

If yes, please specify results. 

33 What do you think the benefits are: 

• to the appraisee? 

• to the appraiser? 

• to the group /team performance? 

• to the organisational performance? 

• other? 

(Textual responses only - see Tables 5, 6, 7 and 8 for analysis) 

34 Do you consider the appraisal scheme to have any disadvantages? If 
yes, what do you consider them to be? 

(See also Table 9 for content analysis of textual responses) 

35 Are there any plans for changes to the scheme? If yes, can you 
indicate what the changes might be? 

(See also Tables 10 and 11 for content analysis of textual responses) 
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Figure 1: Types of further education institutions in the sample 
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Figure 2: Size of further education institutions in the sample, in 
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Figure 3: Position of further education colleges with respect to the 
Investors in People Standard 
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Figure 4: Extent of implementation of appraisal schemes - number 
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Please note - percentages in all of the following figures refer to 
percentage of schemes in the sample 

Figure 5: Year of introduction of appraisal schemes 
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Figure 6: Categories of staff covered by appraisal schemes 
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Figure 7: Extent to which participation in appraisal is compulsory 
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Figure 8: Extent to which appraisal schemes are designed to meet 

VARIOUS PURPOSES 
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Figure 9: Relationship between appraisal and other procedures (in 

TERMS OF IS IT SEPARATE FROM THE PROCEDURE?) 
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Figure 10: How often staff are appraised 
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Figure 11: Extent of interim review as a feature of appraisal 
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Figure 12: Models of appraisal in further education colleges 
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Figure 13: Others having input to the appraisal scheme 
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Figure 14: Time allocation for appraisal 
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Figure 15 : Degree of choice of appraiser 




Figure 16 : Extent and use of appeals procedures 
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Figure 17: Training for appraisal 
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Figure 18 : Types of appraisal forms available for use in schemes 
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Figure 19: Access to the full record of the appraisal interview 
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Figure 20: Extent to which observation of teaching and evaluation 

OF TASKS ARE INCLUDED IN APPRAISAL SCHEMES 
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Figure 21: Extent to which other evidence contributes to the 

APPRAISAL PROCESS 
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Figure 22: Appraisal preparation - extent of pre-meetings and 

EXCHANGE OF DOCUMENTS 
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Figure 23: Location of appraisal interviews 
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Figure 24: Extent to which appraisal schemes are designed to include 

THE IDENTIFICATION, RECORDING AND MONITORING OF SPECIFIC OUTCOMES 
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Figure 25: Level of detail to which appraisal outcomes are recorded 
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Figure 26 : Extent of monitoring of appraisal schemes 
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Figure 27 : Extent of evaluation of appraisal schemes 
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Figure 28 : Extent of cost-benefit analysis of appraisal schemes 
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Figure 29 : Extent of perceived benefits of appraisal schemes 
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Figure 30: Whether or not disadvantages are Perceived 




Figure 31: Whether or not changes are planned 




ERIC 



Yes 

No 

Don’t know 
No response 



Yes 

No 

Don't know 
No response 



DEVELOPING F E 



FEOA report 



Voh 




FEDA publication series 



Developing FE: Volume i 

1. Student tracking 

2. Caseloading 

3. Assessing the impact: provision for learners with learning 
difficulties and disabilities 

4. Adults and GNVQs 

5. On course for next steps: careers education and guidance 
for students in FE 

6. Marketing planning 

7. Managing change in FE 

8. The effective college library 

9. Appraisal in FE - where are we now? 

10. Clarity is power: learning outcomes and course design 



FE Matters: Volume i 

1. Environmental education in FE: part 1 

2. Environmental education in FE: part 2 

3. Towards self-assessing colleges 

4. Colleges working with industry 

5. Evidence for action: papers prepared for FEFCs Learning 
and Technology Committee 

6. Student retention: case studies of strategies that work 

7. Getting the credit: OCN accreditation and learners with 
learning difficulties and disabilities 




92 



DEVELOPING F E 



FEDA report 



95 




8. Moving on from Key Stage 4: the challenges for FE 

9. Monitoring student attendance 

10. Educational Psychologists in FE 

11. Assuring coherence in individual learning programmes 

12. Adult learners: pathways to progression 

13. A real job with prospects: supported employment 
opportunities for adults with disabilities or 
learning difficulties 

14. Transforming teaching: selecting and evaluating 
teaching strategies 

15. Information and learning technology: a development 
handbook 

16. Developing modem apprenticeships 

17. Planning a merger of FE colleges 

18. Tackling drugs together: addressing the issues in the 
FE sector 

19. Security is not an option - learning in a safe environment 

20. Give us the credit: achieving a comprehensive FE 
framework 



FEDA bulletins: Volume i 



1. Developing college strategies for Human Resource 
Development 

2. Enhancing GCE A-level programmes 

3. Impact of voucher schemes on the FE curriculum 

4. Quality assurance in colleges 

5. Maintaining quality during curriculum change 

63 



O 

ERIC 



DEVELOPING FE 



FEDA report 



Voli 




6. Action planning and recording achievement 

7. Implementing modular A levels 

8. Comparing content in selected GCE A levels and 
Advanced GNVQs 

9. Engineering the future 

10. Charters in FE: Making them work 

11. Access to accreditation 

12 Back to the future: modern apprenticeship schemes 

13. Competing for business: colleges and the 
Competitiveness Fund 

14. Information systems: a strategic approach 

15. Strategic approaches to processes, cultures and structures 




94 



DEVELOPING 



FEDA report 



97 









; ; ¥6liiiiw 1 Number 9 

? lM9S {la®b®® 

fi ,■.■■.:■■ i ■. ■ ' 

-■ 

i- i . 

i ■■ 




I- ^ ' 

ir ^Ni 

I'^O 

Sr O 

I me 





as. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 




NOTICE 

REPRODUCTION BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket”). 



